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Executive Summary 


The Quality Assurance Report (QAR) Program, which includes assessment and 
coaching, is a continuous improvement process to support early childhood programs. In 
2022, Missouri Department of Elementary and Secondary Education Office of Childhood 
(DESE OOC) contracted with the Early Childhood Innovation Center (ECIC) at the 
University of Missouri-Kansas City-Institute for Human Development (UMKC-IHD) to 
implement the QAR Pilot Program for a sixth cohort of 250 classrooms in 50 early 
childhood programs. The project was expanded in 2023 by enrolling new programs 
(some from Cohorts 1-5) for a total of approximately 500 classrooms. For all 
classrooms, DESE OOC is using the Classroom Assessment Scoring 
System®(CLASS). This interim report describes the first 18 months of the project. 


Since February 2022, ECIC completed the launch and installation of the project. This 
included assigning or hiring 18 highly qualified staff members, onboarding new staff, 
training staff in the assessment instruments, and training staff in the coaching model. 
This year, ECIC incorporated CLASS affiliate trainers to the QAR team. This action 
directly supports state-wide capacity-building for QAR implementation. Early childhood 
quality specialists (ECQS) are located throughout the state to ensure comprehensive 
program implementation. 


Installation also included contacting and enrolling early childhood programs to confirm 
their participation in the cohort. To date, 88 programs representing 498 classrooms are 
enrolled. ECIC conducted five orientation webinars to introduce the QAR Pilot to 
program leaders and staff. Slight changes to the initial classroom assessment protocols 
include shifting from the CLASS Pre-K assessment instrument to the Second Edition of 
the CLASS Pre-K assessment instrument and adding the CLASS Infant/Toddler and 
CLASS Pre-K Environments tool. Missouri is on the forefront of implementation with 
these instruments since they were just published within the past year. 


Preliminary CLASS data are available for 436 classrooms (85 infant classrooms, 153 
toddler classrooms, and 198 Pre-K classrooms). Overall, scores are slightly higher than 
national norms. Scores are grouped by domains and use a seven-point scale. Scores of 
1 to 2 are considered ineffective, 3 to 5 indicate a mix of effective and non-effective 
interactions, and 6 to 7 indicate effective interactions are consistent. There is one 
domain for infants, two domains for toddlers, and three domains for Pre-K children. The 
average infant domain score for Responsive Caregiving was 4.64. The toddler score for 
Emotional and Behavioral Support was 5.30 and 3.21 for Engaged Support for 
Learning. For Pre-K, Emotional Support averaged 5.69, Classroom Organization 
averaged 5.10, and Instructional Support averaged 2.93. 


Coaching sessions occurred on a regular basis and reports were analyzed. A total of 
1,503 coaching visits were completed with educators since the project began. Coaching 
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sessions mostly occurred in-person (80.2%). The majority of coaching interactions 
focused on building partnerships (40.9%), which is the foundation of Practice-Based 
Coaching. For 29.2% of visits, the focus was on shared goals and action planning. 
Educators’ feedback indicate they are reflecting on their practices (4.48), improving on 
their practices (4.38), and deciding their own goals (4.41). Educators reported less 
efficacy in making progress on their goals (3.94); however, many of them have only 
recently engaged in coaching, so it is expected that goal attainment ratings will improve 
over time. 


Results suggest that using CLASS for the QAR program is a promising approach. It 
appears that the design of the program and the coaching model are scalable, feasible, 
and well-received by program administrators. Baseline data collected and analyzed in 
this work will be used to examine the effects of data-informed coaching. This study can 
be used to inform future efforts regarding quality improvements in early childhood 
programs in Missouri. 


“| feel that participating in the QAR 
program Is very beneficial to the center.” 
- Educator 
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Background 


In December 2019, DESE received a Preschool Development Grant Birth to Five (PDG 
B-5) grant from the U.S. Administration for Children and Families, U.S. Department of 
Health and Human Services (DHHS) and the U.S. Department of Education (DOE). This 
grant provides funding to coordinate a “more effective, high-quality early learning 
system that prepares children to be ready for school.” One activity for the PDG B-5 
grant is to pilot the Quality Assurance Report (QAR), a continuous improvement 
process for early learning programs and to provide families with information about 
quality programs. The Pilot Program is authorized by the Missouri Legislature. The QAR 
employs evidence-based practices with a goal to ensure that all children have access to 
high-quality early childhood care and education (ECCE). The project is designed to 
intentionally include children with disabilities, special health care needs, and those in 
rural areas with limited access to early childhood programs. 


Since 2019, five cohorts of early childhood programs participated in the QAR Pilot 
Project. Their participation included evidence-based classroom environment 
assessments and on-site consultation from early childhood professionals. The Missouri 
Department of Elementary and Secondary Education Office of Childhood (DESE 
OOC)was interested in exploring a different model for the sixth cohort, using the 
Classroom Assessment Scoring System®"(CLASS). CLASS is an observational 
assessment tool and a recognized quality improvement system that measures teacher- 
child interactions. DESE OOC is also interested in testing the efficacy of a model that 
provides assessment and technical support related to program administration, before 
providing supports in classrooms. 


Project Design 


In 2022, DESE OOC contracted with the ECIC to implement the QAR Program for 
Cohort #6 and evaluate its effectiveness. The work plan included assessing 
approximately 250 classrooms in 50 programs. In 2023, DESE OOC expanded the 
contract with the ECIC to implement the QAR project with all currently participating 
programs and additional newly enrolled programs. This work plan involves serving a 
total of 500 classrooms. 


Programs new to the project were encouraged to attend a virtual orientation webinar. 
These webinars were offered at multiple days and times. Webinars outlined timelines, 
project design, and coaching implementation. After the webinar, the programs were 
contacted to begin the coaching process. This includes a packet of information that 
outlines the coaching agreement, information about the assessment tools, and 
information about coaching implementation. The first visit with the teacher included 
reviewing the coaching agreement and coaching model, a focused process of getting to 
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know the teacher, and defining preferred coaching strategies. The second visit with the 
teacher included information regarding assessments, as well as initial goal planning. 


ECIC is conducting a comparative evaluation study to examine the effectiveness of two 
models: 


Model # 1: Classrooms are assessed using the CLASS instrument and teachers 
engage in coaching for continuous quality improvement (CQI) focusing on 
effective teaching practices over a period of 16 months. 


Model #2: In addition to classroom assessments using the CLASS, programs in 
Model 2 are also assessed regarding administrative policies and practices using 
three instruments: the Program Administration Scale®’ (PAS), the Organizational 
Equity Assessment” (OEA), and the Early Childhood Work Environment 
Survey®' (ECWES). Administrators engaged in coaching and technical support 
for CQI at the program level during 2022, focusing on effective administrative 
practices. In 2023, ECIC is re-assessing programs regarding administrative 
policies and practices to measure improvement. If programs show progress and 
meet an established program quality threshold, they receive CQI coaching at the 
classroom level, focusing on effective teaching practices for the remainder of 
2023. If programs do not meet the quality threshold, they will continue to receive 
coaching on administrative practices for the duration of the project. 


For the comparative study, ECIC is using a pre-/post- research design to measure 
improvement over time, with classroom interactions (CLASS scores) as the primary 
outcome variable. Programs were randomly assigned to either Model #1 or Model #2. 
All classrooms, regardless of their model assignment, received initial CLASS 
assessments to establish a baseline measurement and a second assessment (at Time 
2) using the CLASS at the end of the study. The comparative analysis will show 
change-over-time for the two models. A final report will be prepared to describe the 
process and outcomes of the study. 
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ECIC is using the Practice-Based Coaching (PBC) model for structured coaching 
implementation. PBC is a cyclical process for supporting teaching and administrative 
staff's use of effective practices that lead to positive outcomes for children and 
programs. Fundamental to PBC is Establishing the Partnership, an essential element to 
coaching cycle components that ensures collaboration between the coach and coachee. 
The coaching cycle components are: 1) Shared goals and action planning; 2) Focused 
observations; and 3) Reflection and feedback. ECIC is using a hybrid-coaching format, 
with monthly on-site coaching sessions, and monthly virtual coaching visits with 
teachers and administrators. 


Classroom interactions using the CLASS continue to be used, along with an 
environmental assessment using the CLASS Environments. PBC model for structured 
coaching interventions continues to be the method for interactions between early 
childhood quality specialists (ECQS) and educators. 


Process Evaluation 
ECIC Staffing 


ECIC assigned 24 individuals (18.14 FTEs) to work on the QAR Project. Staff roles 
include principal investigator, project coordinator, data analyst, team leads, ECQS, and 
logistical support staff. To expand project services and scope for 2023, the project was 
restructured to include team leads who assist in supervision of the ECQS, as well as the 
addition of eight ECQS. Team Leads were promoted from existing ECQS positions. 


ECQS are responsible for assessing and coaching programs. A total of 14 positions 
were available, with existing specialists filling three of these roles and 11 additional staff 
hired. Applicants were recruited from local, regional, and national networks including 
regional networks, the Missouri Association for the Education of Young Children, 
Missouri Head Start Association, the certified CLASS assessor registry, the American 
Educational Research Association, National Head Start Association, and the National 
Association for the Education of Young Children. These organizations were encouraged 
to forward the recruitment posting throughout the state and across the U.S. Specific 
requirements for the ECQS positions included: 


e Bachelor's degree or higher, 

e Experience in early childhood education, 

e Experience working with adults (specifically training, coaching, or mentoring), 
and 

e Willingness to engage in field-based, remote work (including travel within the 
state). 


Interviews were held virtually. Most specialists started March — April 2023. All specialist 
positions are currently filled, and the project is fully staffed. 
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The QAR staff represent a highly qualified array of educational backgrounds and work 
experiences. Of the five staff that are in leadership positions with the project, two have 
doctoral degrees, one has an educational specialist degree, and two have master’s 
degrees. Of the ECQS, six (43%) have master’s degrees and eight (56%) have 
bachelor’s degrees. They also bring a broad range of experiences to the position: 
school district administration, early intervention, classroom-based instruction, early 
learning program management, music therapy, and non-profit administration. 


Staff Onboarding. In addition to standard ECIC on-boarding processes, staff for the 
ECQS positions participate in a series of on-going staff development activities. 


e Classroom Assessment Scoring System (CLASS) — CLASS training is 
completed as a synchronous, on-line training sessions for staff through 
Teachstone’ initially and then by ECIC affiliate trainers. Staff then completed 
rigorous reliability testing through Teachstone for each of the Infant, Toddler, and 
Pre-K instruments. 


e Practice-Based Coaching - Coaches complete an online, asynchronous training 
on Practice-Based Coaching (PBC)?, with a follow-up two-day training, either in- 
person or virtually. 


Observation Tool Training and Reliability. To maintain fidelity with program 
implementation, ECIC requires ECQS to be trained and achieve reliability in the CLASS 
instruments to conduct observations. All of the ECQS were trained and achieved 
reliability in CLASS as reported in Table 1. They are also required to be trained in PBC 
before they enter the field for coaching duties. 


Table 1. Observation Tool Reliability 


Number of Staff 


Observation Tool : Percent Reliable 
Trained 


CLASS — Infant 18 100% 
CLASS — Toddler 18 100% 
CLASS — Pre-K 18 100% 
CLASS -— Infant/Toddler Environments 18 N/AS 
CLASS — Pre-K Environments 18 N/A‘ 
CLASS — Pre-K Affiliate Trainer 5 100% 
CLASS -— Infant/Toddler Affiliate Trainer 4 100% 


1 Teachstone, Inc., is the owner of the CLASS instrument and exclusively provides products and training 
for observer certification, on-going support, and training for the affiliate model. 
(https://teachstone.com/products-and-trainings/ ) 

2 The Pyramid Model Consortium offers online training in the Practice-Based Coaching model 
(https://www.pyramidmodel.org/ ). The ECIC Leadership Team offers additional training the model for 
application in the QAR Program. 

3 The CLASS -— Infant/Toddler Environments and CLASS — Pre-K Environments were released in 2022 
and tools for establishing reliability are not yet developed. The ECIC is participating in the reliability study 
for these instruments. 
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On-Going Program Staff Development 


To ensure the remote program staff continue in their professional growth and to ensure 
consistency in program implementation, a variety of meetings are scheduled with staff 
on a regular basis. 


e Team Meeting — The QAR project team gathers monthly to discuss program 
implementation. Topics have included human resources reminders, 
scheduling/training updates, coordination of service delivery, and other 
administrative topics. An agenda is provided, and agenda items are solicited prior 
to the staff meeting. 


e Professional Learning Community (PLC) — Project team members meet twice 
monthly to engage in a robust book study. Team members rotate leading the 
PLC meeting. Team members completed the study of Conscious Discipline: 
Building Resilient Classrooms“ in March 2023. Project team members were 
queried in February 2023 to select a topic for the next book study. Of the topics 
generated, ECQS requested further study in culturally competent coaching 
practices. The book Hearing All Voices: Culturally Responsive Coaching in Early 
Childhood was selected and study began in May. 


e Coaching Roundtable — ECQS meet monthly to attend this open-format 
meeting where they bring questions and ideas centering around implementation 
of PBC with educators. This meeting is facilitated by the Team Leads. 


e Conference Attendance — Selected QAR program staff attended and presented 
at conferences to contribute to their professional development and to represent 
the QAR Project to the greater early childhood community. Staff attended 
Conference on the Young Years, Teachstone InterAct, and the National 
Association for the Education of Young Children Professional Learning Institute. 


QAR Information Dissemination 


Several activities for information dissemination about the QAR Program have been 
available throughout the first half of program implementation. 


e Conference on the Young Years (CYY) — The QAR Project Coordinator 
presented two sessions at CYY in March 2023: What is the Missouri Quality 
Assurance Report? and All About CLASS. Each session was attended by 
approximately 35 educators. Also, ECIC hosted an informational booth at CYY. 
Approximately 600 informational fliers about the QAR project were distributed 
and 30 programs submitted applications as a result of this recruitment activity. 


7 | Early Childhood Innovation Center 


Ue 


Institute for 
Human Development 


MO-SECA Leadership Academy — QAR project fliers were included in 
participant packets for the Missouri Supporting Early Childhood Administrators 
(MO-SECA) Leadership Academy in March 2023. Approximately 219 
administrators attended this academy. 


InterAct Conference — The first in-person conference hosted by Teachstone 
was held in April 2023 and attended by approximately 350 individuals from 
across the U.S., including the ECIC QAR project team. The QAR Project 
Coordinator presented a session: Missouri Quality Assurance Report: Using 
CLASS to Improve Teaching Practices. Also, the ECIC QAR leadership met with 
Bridget Hamre (author of the CLASS Instruments) and with Teachstone’s 
research and evaluation team to share about developments in the QAR and 
explore opportunities for collaboration. 


InterAct Now! — The virtual companion to the InterAct conference, InterAct Now! 
was held the first week of May 2023. The Project Coordinator presented two 
sessions at the conference: Missouri Quality Assurance Report: Using CLASS to 
Improve Teaching Practices and Using Innovative Methods to Analyze CLASS 
Data for Targeted Coaching Interventions. 


Program Enrollment 


Eighty-eight early learning programs with 498 classrooms are enrolled in QAR. Of 
these, 107 classrooms continued from Cohorts 1-5, 214 classrooms are enrolled from 
Cohort 6, and 177 classrooms were recruited and enrolled in 2023. Table 2 shows 
details about the programs. 


Table 2. Information about QAR Enrolled Programs 


Number of Programs 88 
Number of Classrooms 498 
Program Size 
Family Child Care 4 
Small (1-5 classrooms) 49 
Large (6+ classrooms) 35 
Region (Classrooms) 
North Central 15 
Northwest 7 
Northeast 9 
Kansas City 58 
Central 77 
St. Louis 214 
Southwest 83 
Southeast 18) 
Average Child Enrollment Capacity 97 
# of Programs Receiving State Subsidy 74 
# of Programs Receiving Head Start Funding 14 
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Outcome Evaluation 
Classroom Assessments 


CLASS Overview. The CLASS is an observational tool used to evaluate the process 
quality of early childhood classrooms. The CLASS focuses on observed teacher-child 
interactions through a series of four short observation rounds (15 — 20 minutes per 
round depending on the instrument used). After each observation round, each 
dimension is scored on a scale of 1 to 7. Scores are determined by scoring indicators 
for each dimension low, mid, or high. CLASS instruments are tailored to different age 
groupings: infant (0-18 months), toddlers (19-35 months), and preschool (3 — 5 years). 
Beginning in 2022, Teachstone published a revised and updated version of the Pre-K 
CLASS tool (2"¢ Edition). The QAR Project used this instrument to assess preschool 
classrooms beginning in 2023, along with the CLASS Infant and CLASS Toddler 
instruments. Table 3 outlines the domains and dimensions for each CLASS instrument. 


Table 3. Domains and Dimensions of the CLASS 


Pre-K — 2"? Edition 
(3-5 years old)*! 
Emotional and Behavioral Emotional Support 
Support e Positive Climate 


e Positive Climate e Negative Climate 

e Negative Climate e Educator Sensitivity 

e Teacher Sensitivity e Regard for Child 

e Regard for Child Perspective 
Perspectives 


e Behavior Guidance 


Engaged Support for Classroom Organization 
Learning e Behavior 
e Facilitation of Management 
Learning and e Productivity 
Development e Instructional Learning 
e Quality of Feedback Formats 


e Language Modeling 


Instructional Support 
e Concept Development 
e Quality of Feedback 
e Language Modeling 
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CLASS Results. A total of 436 CLASS observations were completed in April 2022 for 
Cohort 6 and from March — June 2023 with classrooms from Cohorts 1-5 and newly 
enrolled programs. Of these, 85 Infant classrooms, 153 Toddler classrooms, and 198 
Pre-K classrooms were assessed. There are 62 observations remaining that have been 
deferred to fall 2023 due to classroom and/or program closures (e.g., closed for 
summer, no lead teacher). Preliminary CLASS data show that, overall, scores in 
Missouri programs tend to be higher than the current national averages. 


The relational climate/positive climate and teacher sensitivity dimensions are the 
highest scoring items in the mid-high range, with scores of ranging from 5.09 — 5.82. 
Dimensions regarding implementing concept-related content to children and language 
support were in the low to low-mid range, with scores ranging from 2.58 — 3.56. These 
initial results are consistent with reported national averages. Domain and dimension 
average scores are reported in Table 4. 


a Preliminary CLASS data show that, 


overall, scores in Missouri programs 
tend to be higher than the current 
national averages. 
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Table 4. Class Instrument Preliminary Avera 


CLASS Instrument 


ge Scores 
Average 


Score 


National 
Average 


Responsive Caregiving 4.64 .98 - 
Relational Climate 5.44 0.85 5.04 
Teacher Sensitivit 5.27 1.05 4.75 
Facilitated Exploration 4.10 1.34 3.68 
Early Language Support 3.83 1.20 3.31 

Emotional and Behavioral Support 5.30 0.78 - 
Positive Climate 5.48 0.97 5.03 
Negative Climate* 6.36 1.43 5.30 
Teacher Sensitivity 5.09 1.09 4.33 
Regard for Child Perspectives 4.64 1.03 4.36 
Behavior Guidance 4.59 1.18 4.07 

Engaged Support for Learning 3.21 0.89 = 
Facilitation of Learning and Development 3.56 1.03 3.43 
Quality of Feedback 2a 0.95 - 
Language Modeling 3.24 0.94 222 

CLASS Pre-K (n=198) 

Emotional Support 56g 0.77 - 
Positive Climate 5.82 1.00 5.21 
Negative Climate’ 6.82 0.31 6.37 
Teacher Sensitivity 5.40 1.00 4.34 
Regard for Student Perspective 4.80 1.02 4.36 

Classroom Organization 5.10 1.01 - 
Behavior Management 5.37 1.09 4.94 
Productivity 3.07 1.13 5.41 
Instructional Learning Formats 4.60 1.04 4.57 

Instructional Support 2.93 91 - 
Concept Development 2.58 88 2.69 
Quality of Feedback 2.89 .94 2.87 
Language Modeling 3.25 .98 2.85 


CLASS Environment. Beginning in 2023, new programs also had the CLASS 
Environment assessments conducted in their classrooms. The CLASS Environment 
measure is a supplemental tool designed for use in conjunction with the CLASS 
observation, which describes and measures features of the environment that can 
support effective interactions. The CLASS Environment assessments are divided into 
two instruments: Infant-Toddler (birth — 35 months) and Pre-K - K (36 months — 
Kindergarten). The Infant-Toddler tool consists of 32 items (14 in Emotional and 
Behavioral Support and 18 in Engaged Support for Learning) and the Pre-K-K tool 
consists of 29 items (10 in Emotional Support, 8 in Classroom Organization, and 11 in 


4 Scores for these dimensions are reverse-coded for this item and have been adjusted to match the 1-7 


scale. 
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Instructional Support). ltems,scored on a scale of 1 to 3 (1 — no, none. never, 2 — few, 
some, limited, and 3 — most, many, varied), correlate with the CLASS dimensions for 
each tool. 


Most Environment scores averaged in the “few, some, limited” range. The overall infant 
classroom average was 2.15, overall toddler average was 2.29, and overall preschool 
average was 2.46. Table 5 shows the results of the domain and dimension average 
scores for the CLASS Environment assessments for infant, toddler and Pre-K 
classrooms. Further comparison of scores with CLASS scores and implications for 
practice will be included in the final report. 


Table 5. CLASS Environment Average Scores 
CLASS Environment Instrument Average Score 


| Emotional and Behavioral Support 2.24 
Arrangements 2.62 0.65 
Displays 2.68 0.78 
Materials 219 0.53 
Arrangements/Displays/Materials 1.84 0.84 

Emotional and Behavioral Support 2.08 0.83 
Arrangements 2.66 0.61 
Displays 1.00 0.00 
Materials 2.09 0.83 
485 Environment Toda Clas 1.71 0.72 

Emotional and Behavioral Support 2.34 0.80 
Arrangements 2.65 0.67 
Displays 2.81 0.80 
Materials 2.86 0.44 
Arrangements/Displays/Materials 2.02 0.76 

Emotional and Behavioral Support 2.26 0.82 
Arrangements 2.76 0.48 
Displays 1.42 0.69 
Materials 2.29 0.81 
Arrangements/Displays/Materials 2.01 0.83 

CLASS Environment Pre-K (n=99) | 2.46 | 0.74 

Emotional Support ASS 0.68 
Arrangements 2.61 0.67 
Displays 2.47 0.66 
Materials 2.98 0.14 
Arrangements/Displays/Materials 2.34 0.74 

Classroom Organization 200 0.68 
Arrangements 2.79 0.81 
Displays 2.02 0.51 
Materials 2.77 0.81 

Instructional Support 2.28 0.81 
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CLASS Environment Instrument Average Score SD 
Displays 1.92 0.85 
Materials 22t 0.81 
Displays/Materials 2.41 0.76 

Coaching 


Program administrators and teachers receive individual feedback, delivered by their 
assigned ECQS. The ECQS meet with the program director for center-based educators 
to debrief assessment scores, review coaching expectations through review of the 
Coaching Agreement, and share how educator goals will be shared. Specialists then 
engage in coaching with educators using the PBC model. The initial educator visit 
includes discussion of the Coaching Agreement and the implementation of different 
coaching strategies, gathering input into preferred coaching strategies to guide 
coaching activities. Subsequent visits involve sharing individual classroom assessment 
scores, completing individual needs assessments on identified teaching practices, and 
beginning the PBC coaching cycles starting with goal and action planning, following with 
the remain components of focused observation and reflection and feedback. 


Practice-Based Coaching. 
Coaching implementation Figure 1. Practice-Based Coaching Component 
for programs in Cohort 6 Implementation (n=1,503) 

began in August 2022. With 
the addition of all QAR 
participants in 2023, a total 
of 1,503 coaching visits 
have occurred, totaling 
731.5 hours of coaching 
with classroom educators. \ 

Coaching sessions mostly P 

occurred in-person oo oung> 
(80.2%). Coaching 19.7% 
activities are reported for 
each coaching interaction 
with an educator. 


29.2% 7 40.9% 


= Establishing the Partnership = Focused Observation 


= Reflection and Feedback = Shared Goals and Action Planning 


The majority of coaching interactions focused on Establishing the Partnership (40.9%). 
This follows the expectation for the Practice-Based Coaching model to build 
relationships with educators as the first step of the coaching relationship. (See Figure 


1.) 


Coaches reported which coaching strategies they used during coaching sessions. 
Multiple methods can be used during one session. Coaching strategies varied greatly. 
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The intentional focus on relationships (42.8%) was the most frequent component of 
coaching, followed by the ongoing use of an action plan (31.9%) and reflective 
conversations (27.4%). 


Coaches also self-rated how successful they felt each component of the PBC was 
implemented at each visit. On a scale of 1 (not at all successful) to 5 (very successful), 
coaches rated overall coaching a 4.19 or mostly successful. 


An essential component of Practice-Based Coaching is for programs to self-select their 
own target goals. For infant educators, goals regarding early language support were 
most common (55.9%), followed by facilitated exploration (24.6%) and relational climate 
(16.4%). For toddler educators, most educators chose a goal in the language modeling 
(33.3%), behavior guidance (30.2%), or facilitation of learning and development (22.3%) 
dimensions. In preschool classrooms, educators most frequently set goals in the 
behavior management (29.9%) and concept development (28.6%) dimensions. 


Coaching Session Feedback. A Coaching Session Feedback Survey was created to 
obtain on-going feedback from program participants about their experiences in the 
program. This brief survey is designed to be completed at the conclusion of each 
coaching visit in less than five minutes. Respondents also have the option to request 
contact by the project coordinator and can provide any other feedback to the QAR team. 
Respondents rated themselves on the following questions, on a scale of 1 — No/Not At 
All to 5 - Extremely: 


e To what degree do you feel you were able to decide the goals and steps you are 
going to work on? 

e To what degree do you feel you are making progress on your goal? 

e The interactions with my coach helped me to improve my practice? 

e The discussion with my coach prompted me to reflect about my practice in a 
meaningful way? 


To date, 630 surveys have been gathered (42%). Overall, program participants feel that 
they are reflecting on their practices (4.48), improving on their practices (4.38), and 
deciding on their own goals (4.41). Educators reported less efficacy in making progress 
on their goals (3.94); however, many of the educators currently served through the 
project have only recently engaged in coaching, so this is not an unexpected finding. 


Comments from educators were also gathered on feedback surveys. A sampling of 
comments includes: 


e “She made this process much less stressful than expected.” 

e “lam looking forward to receiving more help and feedback on ways to help teach.” 

e ‘“[Coach] has been so patient and supportive working with me on the administrative 
scale. | have loved working with her.” 
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e “lam starting Monday as a coach for Early Head Start. [Coach] has been a great role 
model in coaching as well as CLASS. Teachstone has been a valuable resource to me 
as well!” 

e “l feel that participating in the QAR program is very beneficial to the center.” 


Comparison Study 


The administrative coaching provided through the comparison study with Cohort 6 
concluded in April 2023. Some programs chose to end coaching interventions earlier 
between January — April 2023. Time 2 assessments using the Program Administration 
Scale began in May 2023 and will conclude by the end of June 2023. Time 2 CLASS 
assessments for these programs will be conducted in Spring 2024. Comparative data 
analysis will be performed thereafter. 


Conclusions 


Examination of the process for developing and launching the QAR program suggests 
the project is meeting objectives in a comprehensive and thorough way. Although hiring 
and onboarding staff was a longer process than anticipated, the QAR staff are highly 
qualified, well-trained, and conduct their assessment and coaching activities efficiently 
with fidelity to the model. Rigorous training protocols and meaningful, on-going 
professional development were useful for building knowledge, skills, and confidence of 
the staff. The ECIC QAR leaders were successful in navigating the project launch with 
Cohort 6 and seamlessly conducted the 2023 expansion in a timely and effective 
manner. While not all of the Cohort 1-5 programs were retained, the 107 classrooms 
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that are continuing in QAR are adapting to the new instruments and indicate that the 
PBC approach to coaching is useful. 


Communication and carefully constructed protocols are hallmarks of the project 
success. Written communication tools are incorporated into the standard workflow for 
onboarding educators and their administrators. The orientation webinars provide 
participants with context about the program and specific details that set the stage for 
effective implementation. Written materials and systematic electronic communication 
provide support for each participant’s onboarding; however, the most important 
communication vehicle is personal communication with the Quality Specialists as they 
conduct initial assessments and build trusting relationships with the coachees. 


The robust baseline data collected thus far in the project are useful for understanding 
the nature of quality early learning programs in Missouri and will inform the further 
development of QAR. Feedback from both coaches and program participants suggest 
that educators are deeply considering aspects of their practice and beginning to make 
progress on goal attainment. 


Assessment and coaching continue to operate at a steady pace. Coaching 
implementation will continue to be analyzed to determine patterns and potential areas 
for professional development of program staff and educators. In collaboration with 
OOC, ECIC is preparing for adding 500 classrooms to the project as projected for Fall 
2023. Recruitment for these classrooms will begin in August 2023. One lesson learned 
is that the timing and scheduling of assessment and coaching is important for some 
programs. ECIC will anticipate summer closings and work closely with individual 
programs to ensure that QAR can be adapted to participants’ unique situations. 


Information dissemination is an essential component of the QAR for multiple 
stakeholders in Missouri and across the U.S. Because the OOC and ECIC are 
choosing to incorporate cutting-edge assessment tools and methods, the project is 
attracting the attention of national thought leaders and researchers as an innovative 
approach. For example, one research opportunity is the examination of the relationship 
between CLASS and CLASS Environment scores and how these interactions could 
impact coaching and program improvement. The ECIC will continue to seek 
opportunities to message about the work and valuable findings from the process. 


The ECIC is grateful for the opportunity to partner with OOC in developing an early 
childhood quality improvement system for Missouri. The ECIC leadership is committed 
to providing QAR data and grounded insight to inform the process of creating a system 
that recognizes quality programs and practices. Over the coming months, the QAR 
project team and a group of identified stakeholders will support the OOC in designing 
additional components to the QAR project. This development will continue to facilitate 
the goal of state-wide program implementation in fall of 2024. 
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Appendices 


A. Program Information Form 


Page I 


Missouri QAR Program Information 2022 


Welcome to the Missoun Quality Assurance Report (QAR) Cohort #6. Please complete the following questionnaire to 
help us know more about your program. If you have any questions, please do not hesitate to contact Melissa Newkirk 
at newkirkm@umkc.edu. 


Contact Information 


Program Name: 
DVN: 

Street Address: 
City: 

Zip Code: 
County: 


State © Missouri 


Phone Number: 

Director First Name: 

Director Last Name: 

Director Email: 

Alternate Contact First Name: 
Alternate Contact Last Name: 


Alternate Contact Email: 


07/29/2022 9:20am projectredcap. org RE DCap 
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Program Information 


What ages does your program serve? 


Please indicate the options your program provides: 


What are your program's hours? 

What is your program's enrollment capacity? 

How many children are currently enrolled part-day? 
How many children are currently enrolled full-day? 
Number of children whose primary language is not the 
language spoken at the program: 


Number of paid employees who work at the program: 


How many administrative staff are employed at your 
program full-time? 


How many administrative staff are employed at your 
program part-time? 


How many teaching staff are employed at your program 
full-time? 


How many teaching staff are employed at your program 
part-time? 


How many support staff are employed at your program 
full-time? 


How many support staff are employed at your program 
part-time? 


How many administrative staff have left the program in 
the last 12 months? 


07/29/2022 9:20am 
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UO Infants (birth - 11 months) 

0 Teddlers (12 - 36 months) 

0 Preschool! (36 months - 5 years) 
0 School-age {5 - 12 years) 


0 Part-day (2-4 hrs.) 
OO School-day (4-8 hrs.) 
OO Full-day (>8 hrs.) 

C Before/After School 


((35 hrs/wk or more)} 
{(10-34 hrs/wk) 
((35 hrs/wk or more)) 
((10-34 hrs/wk)) 
((35 hrs/wk or more)} 


{(10-34 hrs/wk)) 
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How many paid support staff have left the program in 
the last 12 months? 


How many paid teaching staff have left the program in 
the last 12 months? 

Does the director hold an administrator credential? 
Please list the type of administrator credential and 


who it was issued by: 


Please select your program type (select one} 


What is the program licensing status? 


Does the program receive state subsidy? 
Number of children enrolled who receive subsidy: 
Does the program receive Head Start funding? 
Does the program receive state pre-kindergarten 


funding? 

Is the program sponsored by a faith-based 
organization? 

Is the program accredited by NAEYC? 
What is the date the accreditation expires? 


Is the program accredited by another accrediting body? 


Name of accrediting body: 


07/29/2022 9:20am 
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© Yes 
© No 


© for-profit - private proprietary or partnership 

© for-profit - corporation or chain (e.g., 
KinderCare) 

© for-profit - corporate sponsored {e.g., Bright 
Horizons Family Solutions) 

© private nonprofit - independent 

© private nonprofit - affiliated with a social 
service agency or hospital 

©) public nonprofit - sponsored by federal, state, or 
local govemment 

© college or university affiliated 

© military-sponsored 

©) public school 


© exempt 
© license-exempt 
©) licensed 


© Yes 
© No 


© Yes 
© No 


©) Yes 
© No 


© Yes 
© No 


© Yes 
© No 


© Yes 
© No 
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What is the date the accreditation expires? 


Classroom Information 


How many groups (classes of children) does the program 
have? 

Classroom 1 Name: 

Teacher 1 Name: 

Teacher 1 Classroom Role: 
Teacher 2 Name: 

Teacher 2 Classroom Role: 
Teacher 3 Name: 

Teacher 3 Classroom Role: 
Teacher 4 Name: 

Teacher 4 Classroom Role: 
Number of Children Enrolled: 
Age of Youngest Child Enrolled: 


Age of Oldest Child Enrclled: 


Please select the days this classroom is open: 


Hours open: 


Do you have another classroom to add? 


07/29/2022 9:20am 
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© Teacher 
©) Assistant Teacher 


©) Teacher 
©) Assistant Teacher 


©) Teacher 
©) Assistant Teacher 


© Teacher 
© Assistant Teacher 


OJ Monday 

OO Tuesday 

OO Wednesday 
O00 Thursday 
OO Friday 


© Yes 
© No 


projectredcap. org 
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Classroom 2 Name: 

Teacher 1 Name: 

Teacher 1 Classroom Role: 
Teacher 2 Name: 

Teacher 2 Classroom Role: 
Teacher 3 Name: 

Teacher 3 Classroom Role: 
Teacher 4 Name: 

Teacher 4 Classroom Role: 
Number of Children Enrolled: 
Age of Youngest Child Enrolled: 
Age of Oldest Child Enrolled: 


Please select the days this classroom is open: OO Monday 
OJ Tuesday 
CJ Wednesday 
O00 Thursday 


OO Friday 
Hours open: 
Do you have another classroom to add? © Yes 
© No 
Classroom 3 Name: 


Teacher 1 Name: 


Teacher 1 Classroom Role: 


07/29/2022 9:20am projectredcap. org REDCap’ 
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B. Orientation Webinar Feedback Survey 


Missouri QAR Cohort #6 - Orientation 
Webinar Evaluation - Model 2 


Thank you for attending the QAR Orientation Webinar. Please complete this short survey about your 
participation in the webinar. We appreciate your feedback. 


1. What date did you attend the webinar? 


Please input date (M/d/yyyy) & 
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2. About the webinar: 


Strongly Strongly 
Disagree Disagree Neutral Agree agree 


The webinar 
answered my 


questions O O O O '* 


about the 
QAR project. 


The webinar 
dates and 
times offered 


were O O O O O 


convenient 
for my 
schedule. 


| feel 
prepared to 
start the QAR O O O O O 


process. 


The webinar 
facilitator 


answered O O co O O 


questions 
presented. 


The 
information 
presented O O O O O 


was useful. 
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3. The orientation webinar increase my knowledge of the following: 


Strongly Strongly 
Disagree Disagree Neutral Agree agree 


Missouri 
Quality 


Assurance 

Report (QAR) O O O O O 
- Cohort #6 

Models 


Classroom 
Assessment 


Scoring O O O O O 


System 
(CLASS) 


Program 


Administratio O O O O O 


n Scale (PAS) 


Early 
Childhood 


ak O O O O O 


Survey 
(ECWES) 


Early 
Childhood 


Organization 
al Equity O O O O O 
Assessment 


(OEA) 


Practice- 
Based 
Coaching O O O O O 


(PBC) 
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4. What improvements could we make to the webinar to be more helpful? 


5. Are there additional support you need to start the QAR process? 


6. | have additional questions and would like to be contacted. Please provide contact 
information. 


Pe 


This content is neither created nor endorsed by Microsoft. The data you submit will be sent to the form owner. 


Microsoft Forms 
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C. Coaching Agreement — Administrators 


Missouri Quality Assurance Report Administrator Coaching Agreement 


This agreement is between: at and 
(administrator) (program) 
at 
(coach) (organization) 


|, as the Coach, agree to: 


Be respectful and supportive of the administrator by being on-time and prepared for coaching 
sessions. 


Develop goals and action plans with the administrator based on needs that both the 
administrator and coach have identified and agreed upon. 


Schedule, plan, and facilitate coaching sessions. 


Provide support within the Practice-Based Coaching framework model by: 
1. Sharing resources, 

2. Providing models of effective administrative practices, and 

3. Providing supportive and constructive feedback. 


Remain supportive rather than evaluative. 


|, as the Administrator, agree to: 


Actively engage in coaching sessions by: 

Assessing my strengths and needs, 

Asking questions, 

Sharing pertinent information, 

Reflecting, listening, and identifying goals, and 

Collaborating with the coach to determine steps for achieving goals. 


OV ger ee 


Be open to being sharing information and receiving feedback. 
Be open to engaging in a continual process of changing and learning. 
Apply and analyze new administrative practices with the support of my coach. 


Be on time and prepared for coaching sessions. 


Coach’s Signature Date Administrator’s signature Date 


Rev. 5/2022 
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D. Coaching Agreement — Teachers 


Missouri Quality Assurance Report Teacher Coaching Agreement 


This agreement is between: at and, 
(teacher) (program) 
at and, 
(coach) (organization) 
at 
(director) (program) 


|, as the Coach, agree to: 


Be respectful and supportive of the teacher by being on-time and prepared for coaching 
sessions. 


Develop goals and action plans with the teacher based on needs that both the teacher and 
coach have identified and agreed upon. 


Schedule, plan, and facilitate focused observations (including video recording) and coaching 
sessions. 


Provide support within the Practice-Based Coaching framework model by: 
1. Sharing resources, 

2. Providing models of effective teaching practices, 

3. Observing current teaching practices, and 

4. Providing supportive and constructive feedback. 


Remain supportive rather than evaluative. 


Provide feedback to program administration of current goals. 


|, as the Teacher, agree to: 


Rev. 7/2022 


Actively engage in coaching sessions by: 

Assessing my strengths and needs, 

Asking questions, 

Sharing pertinent information, 

Reflecting, listening, and identifying goals, and 

Collaborating with the coach to determine steps for achieving those goals. 


ne Ww hs 


Be open to being observed (including video) and receiving feedback. 
Be open to engaging in a continual process of changing and learning. 
Apply and analyze new teaching practices with the support of my coach. 


Be on time and prepared for coaching sessions. 
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|, as the Program Director, agree to: 
Provide support to both the coach and coachee through check-ins. 


Address any questions regarding progress related to coaching goals to both the coachee and 
coach. 


Acknowledge that coaching data and activities are not used for supervision and performance 
evaluation and the coach will share teacher goals with me. 


Ensure staff coverage for the coachee and coach meetings. 


Coach’s Signature Date Teacher’s signature Date 
Director’s Signature Date 
Rev. 7/2022 
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E. Coaching Report — Administrators 


Early Childhood QAR Coaching Sessions Report - 
Administrators 


Piesse complete the sureey bec. 


Thank you! 


Hame of Coach 


Name of Progam 


Date of scheduled coaching session 


Wes the scheduled coaching session in-person or 4] in-person 

virbualy {I Wirtud 

Chd the cosching session happen? yes 
ae 


Hoe many mmubes for Establishing the Partrershipt 
How many minutes for Foosed Obserrstion (Problem 
SHiving Discussioni? 

Hos many mmubtess for Reflection and Fesschesck 7 


Hie many minutes for Sared Goals arid Adien Manning? 


This Sum shoud be the coral murmber of riniutes of 
Tibertire Session, Bnet, please go beck and adjust 
tm, 
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Which coaching strategies were used during this 
session? 


If other, please explain. 


Goal planning focused on which assessment tool and 
domain? 


Which PAS domain? 


Which OEA domain? 


Which ECWES domain? 


What was the status of a previous goal reviewed? 


Overall, to what degree did you think the session was 
successful? 


To what degree did you think Establishing the 
Partnership was successful in this session? 


07/29/2022 9:25am 
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OO Performance feedback 

OO Collaborative progress monitoring 
0 Ongoing use of an action plan 

0 Help with materials and resources 
O Intentional focus on relationships 
OJ Other 


CO Program Administration Scale (PAS) 
OJ Organizational Equity Assessment (OEA) 
OO Early Childhood Work Environment Survey (ECWES) 


(© Human Resources Development 
© Personnel Cost and Allocation 

©) Center Operations 

©) Child Assessment 

© Fiscal Management 

© Pregram Planning and Evaluation 
© Family Partnerships 

©) Marketing and Public Relations 
©) Technology 


© Organizational Commitment to Equity 

©) Resource Allocation/Budgeting 

©) Organizational Climate/Culture 

©) Outreach/Communication/Family Engagement 
© Human Resources Practices 

© Data Use 

©) Community Collaboration 


© Collegiality 

© Professional Growth 
©) Supervisor Support 
© Clarity 

© Reward System 

©) Decision Making 

©) Goal Consensus 

© Task Orientation 

©) Physical Setting 

© Innovativeness 


© Achieved the goal by meeting the specified criteria 

© Making progress will keep implementing the action 
plan 

©) Made changes to the plan by revising the goal or 
changing the action steps 


©) Not at all Successful 
©) A Little Successful 

© Somewhat Successful 
© Successful 

© Very Successful 


©) Not at all Successful 
©) A Little Successful 

©) Somewhat Successful 
©) Successful 

© Very Successful 
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To what degree do you think Focused Observation 


{Problem Solving Discussion) was successful in this 


session? 


To what degree do you think Shared Goals and Action 


Planning was Successful in this session? 


To what degree do you think Reflection and Feedback 


was successful in this session? 


©) Not at all Successful 
© A Little Successful 

© Somewhat Successful 
©) Successful 

©) Very Successful 


©) Not at all Successful 
© A Little Successful 

© Somewhat Successful 
© Successful 

© Very Successful 


© Not at all Successful 
© ALittle Successful 

© Somewhat Successful 
© Successful 

©) Very Successful 


Did you provide resources to the coachee? 


OYes ONo 


If yes, what resources did you provide? 


What was the reason the coaching session did not 


happen? 


If other, please explain. 


Comments 


07/29/2022 9:25am 
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0 Technology failure 

0 Unavoidable schedule change 

CO Personal issues - coachee 

OO Personal issues - coach 

O00 Scheduled time off 

(0 Coachee had to "staff-in" a classroom 
OJ Other 
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F. Coachee Feedback Survey 


a 7 Page 1 
QAR Coaching Session Feedback Survey 
Thank you for completing this brief survey te help us gather information about the Missoun QAR Pilot Project for 
continuous quality improvement. Your specific responses will help to improve coaching services offered to 
administrators and teachers engaging in coaching activities. 
Prog-am Narre: 
orrzar2o22 S ern pojetreceas cry REDCap 
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faga2 

Coach Name: 
Role for this coaching sessicer: © Program Administrator 

© Classpom Teacher 
Dete of Coaching Vnit: 
During this sessice!, to whet degree do you feel you 
were able to decide the goals and steps you are gong 
to work or: Noe Ar Al Sore Corr plenahy 


Place areay as the ace cory 


During this sessicer, to whet degree do you feel you 
are meking progress an your goal. No Progress Some Mrogess Met My Gow 


lace areay as the acee more 


Dunng ths session, the interactions mth my coach Nor ar ae 

helped me to improwe my practi ce Helpful Somewhat Helpful = Extrersty Helpful 
race area an the aca bony 

Oid today’s coaching session indude reflection and O'tes 

feecback? No 

Dunng ths session, the discussion sath my coach 

prompted me to refect about my practice ina 

meaningful way. {nia alowed if refiection and Nor ara Somewiet Estremehy 

feedback were mot part of my coaching sessor) Mesningtu Nemingu Neoriretul 
Wace BEY ON INS Some DOR) 

Qwould like to be contacted te discuss today’s wat Ores ONo 

wth the project coordinator, 

Pease provide cortact information (email or 

telephone): 

Do you Nave any other feedback you would like to 

provide? 

OT2SN IB worcretopers §=AREDCap 
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